Group comparisons in logit and probit using predicted probabilities1

The comparison of groups in regression models for binary outcomes is complicated by an identification problem inherent in these models. Traditional tests of the equality of coefficients across groups confound the magnitude of the regression coefficients with residual variation. If the amount of residual variation differs between groups, the test can lead to incorrect conclusions (Allison 1999). Allison proposes a test for the equality of regression coefficients that removes the effect of group differences in residual variation by adding the assumption that the regression coefficients for some variables are identical across groups. In practice, a researcher is unlikely to have either empirical or theoretical justification for this assumption, in which case the Allison’s test can also lead to incorrect conclusions. An alternative approach, suggested here, uses predicted probabilities. Since predicted probabilities are unaffected by residual variation, tests of the equality of predicted probabilities across groups can be used for group comparisons without assuming the equality of the regression coefficients of some variables. Using predicted probabilities requires researchers to think differently about comparing groups. With tests of the equality of regression coefficients, a single test lets the researcher conclude easily whether the effects of a variable are equal across groups. Testing the equality of predicted probabilities requires multiple tests since group differences in predictions vary with the levels of the variables in the model. A researcher must examine group differences in predictions at multiple levels of the variables often requiring more complex conclusions on how groups differ in the effect of a variable. 1I thank Paul Allison, Ken Bollen, Rafe Stolzenberg, Pravin Trivedi, and Rich Williams for their comments.